From MAP to Marginals: Variational Inference in Bayesian Submodular Models

Submodular optimization has found many applications in machine learning andbeyond. We carry out the first systematic investigation of inference in probabilis-tic models defined through submodular functions, generalizing regular pairwiseMRFs and Determinantal Point Processes. In particular, we present L-FIELD, avariational approach to general log-submodular and log-supermodular distribu-tions based on suband supergradients. We obtain both lower and upper boundson the log-partition function, which enables us to compute probability intervalsfor marginals, conditionals and marginal likelihoods. We also obtain fully factor-ized approximate posteriors, at the same computational cost as ordinary submod-ular optimization. Our framework results in convex problems for optimizing overdifferentials of submodular functions, which we show how to optimally solve.We provide theoretical guarantees of the approximation quality with respect tothe curvature of the function. We further establish natural relations between ourvariational approach and the classical mean-field method. Lastly, we empiricallydemonstrate the accuracy of our inference scheme on several submodular models.